Web Survey Bibliography
Survey research has historically relied on a probabilistic model to underlie its sampling frame. With rare exception online research is non
‐probabilistic. Research without the safety net of a probabilistic frame raises all kinds of alarms. Challenges as to the reliability of online research has become a growing crescendo as the ‐probabilistic nature of online research has become evident. However, not all sampling frames must be probabilistic. Unfortunately, no such standard metrics exist to track reliability in online sampling. In fact, whether they are access panels or social networks there are no standardized means of balancing panels or even comparing them. To confound the situation the commercially used convenience panels are vastly different from each other (Gittelman and Trimarchi, CASRO Panel Conference, February 2009, paper available). These differences are so far reaching that those who elect to use these sample sources are not only without a safety net, they are at considerable professional risk. We have completed analysis of eighteen American panels and have found that respondent aging, frequency of professional responders, other satisficing behaviors as well as dramatic differences between sociologic, psychographic and buying behavior segmentations make for a cacophony of differences seemingly impossible to correct. ‐panel comparisons themselves are rare with data from a very few having been presented on any scale. ‐liners, invalids, inconsistencies, etc.] for which we have developed standard quantitative measures, and (2) a mechanism for developing a family of sampling standards based upon segmentation by key variables such as, but not limited to, media, purchasing and psychographics. It is the new availability of global data that allows us to present universal standards that help us meet necessary requirements that are our focus in this conference. In addition, to measuring performance, we believe that there are three key requirements for standard panel metrics including: (1) the ability to capture panel performance variations consistent with the differing needs of sample users, (i.e. a broadcasting company might wish to anchor its sampling frame to media segments); (2) The ability to create a data base that is retrospective in that new sample sources can be added to the database without repeating the analysis and (3) a focus on indices that are pragmatic in their measure (i.e. We always view buying behavior as the most pragmatic.)
non
In this study we will present the results of an extensive global study covering forty countries. Within each country panels will be compared using a 17 minute questionnaire, 400 completes per panel. We hope to present five or more providers per market. No such extensive comparison has been done on a global basis. In fact, inter
Preliminary data (24000 interviews) shows evolutionary trends in convenience panel development. Between panel differences appear more extreme in the United States than in other markets.
We are proposing two sets of practices: (1) using panel performance metrics [professionals, speeders, straight
In this talk, we propose to use segmentation analysis as a new metric that will allow us to anchor online data in a new non probabilistic sampling frame. It is the existence of global data that gives us a rare opportunity to experiment with this new methodology. Our goal is to use segmentation in each country to create a fingerprint that can be consistently maintained by blending panels. By minimizing the variability from the segments through optimization and panel combination we will establish a means for stabilizing online data irrespective of the panels and sourcing modes from which they draw their origin. We cannot stabilize online data unless we provide it with a reference point to anchor itself; the segments are that anchor. As the sourcing models continue to shift, panels will age and shift with them; we need a reliable anchor that rises above these problems. It is essential that we explore tools to measure these changes. Without a means of comparison we cannot expect to measure drift nor can we expect to have a platform for predicting the future. We do not profess to be on the road to a new probabilistic framework but rather a platform for comparison and continuity. We believe that there is a theoretical population online that can serve this purpose. Using the database we have gathered that includes respondents from over 160 global panels (64,000 interviews) distributed among 40 global markets we shall introduce new methods to build “perspective”.
Based on this we will use our segmentation models as a means of creating a “convenience” sampling frame by averaging segments into a “Grand Mean.” Using optimization models we will select convenience panels that best reflect the grand mean and the proportions by which they best fit together. We shall give evidence for the efficiency of these strategies.
Conference homepage (abstract)
Web survey bibliography (281)
- Overview: Online Surveys; 2017; Vehovar, V.; Lozar Manfreda, K.
- Standard Definitions Final Dispositions of Case Codes and Outcome Rates for Surveys; 2016
- Retrospective Measurement of Students’ Extracurricular Activities with a Self-administered Calendar...; 2016; Furthmueller, P.
- Pitfalls, Potentials, and Ethics of Online Survey Research: LGBTQ and Other Marginalized and Hard-to...; 2016; McInroy, L. B.
- Computer-assisted and online data collection in general population surveys; 2016; Skarupova, K.
- A Statistical Approach to Provide Individualized Privacy for Surveys; 2016; Esponda, F.; Huerta, K.; Guerrero, V. M.
- Social Media Analyses for Social Measurement; 2016; Schober, M. F.; Pasek, J.; Guggenheim, L.; Lampe, C.; Conrad, F. G.
- Doing Surveys Online ; 2016; Toepoel, V.
- An Overview of Mobile CATI Issues in Europe; 2015; Slavec, A.; Toninelli, D.
- Utilizing iPads in the Field; 2015; Kiser, P.
- Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 2015; 2015
- The Web Survey Revolution ; 2015; Murray, D.
- Methodology of the RAND Mid-Term 2014 Election Panel; 2015; Carman, K. G; Pollack, S.
- 28 Questions to Help Buyers of Online Samples; 2015; Cape, P. J.; Phillips, A.; Baker, R.; Cooke, M.; Ribeiro, E.; Terhanian, G.
- Ethical decision-making and Internet research 2.0: Recommendations from the AoIR ethics working committee...; 2015; Markham, A.; Buchanan, E. A.
- Doing online research involving university students with disabilities: Methodological issues; 2015; De Cesarei, A.; Baldaro, B.
- Exploring ethical issues associated with using online surveys in educational research; 2015; Roberts, L. D.; Allen, P. J.
- An Introduction to Survey Research; 2015; Cowles, E. L.; Nelson, E.
- Ethical issues in online research; 2015; James, N.; Busher, H.
- Leading Edge Insights: Foundations of Quality 2.0; 2014; Fuguitt, G.
- Methods and systems for managing an online opinion survey service; 2014; Mcloughlin, M. H., Seton, N., Blesy, K.
- Recent Books and Journals in Public Opinion, Survey Methods, and Survey Statistics; 2014; Callegaro, M.
- Undisclosed Privacy: The Effect of Privacy Rights Design on Response Rates; 2014; Haer, R., Meidert, N.
- Tailoring mode of data collection in longitudinal studies; 2013; Kaminska, O., Lynn, P.
- How do we Know Cognitive Interviewing is Any Good?; 2013; Willis, G. B.
- Quality of Web surveys; 2013; Revilla, M.
- Experiments in Obtaining Data Linkage Consent in Web Surveys ; 2013; Sakshaug, J. W., Kreuter, F.
- Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical...; 2013; Giesen, D., Bavdaz, M., Loefgren, T., Raymond-Blaess, V.
- Internet as a new source of information for the production of official statistics. Experiences of Statistics...; 2013; Heerschap, N.
- A standard with quality indicators for web panel surveys: a Swedish example; 2013; Nyfjaell, M.
- How Mobile Stacks Up to Traditional Online: A Comparison of Studies; 2013; Knowles, R.
- How to make your questionnaire mobile-ready; 2013; Cape, P. J.
- Phish Rising: How Internet Criminals are Undermining the Viability of Online Survey Research…and...; 2013; Kunovic, K.
- Self-Reported Participation in Research Practices Among Survey Methodology Researchers; 2013; Perez-Vergara, K., Smith, C., Lowenstein, C., Ozonoff, A., Martins, Y.
- Ethics, privacy and data security in web-based course evaluation; 2013; Salaschek, M., Meese, C., Thielsch, M.
- Beyond methodology - some ethical implications of "doing research online"; 2013; Heise, N.
- Code Comparison; 2012
- Evaluation procedures for Survey questions; 2012; Saris, W. E.
- Transparency, Access and the Credibility of Survey Research; 2012; Lupia, A.
- Anonymity and Confidentiality; 2012; Tourangeau, R.
- Cognitive Evaluation of Survey Instruments: State of the Science (Art?) and Future Directions; 2012; Willis, G. B.
- How to provide high data quality in online-questionnaires: Setting guidelines in design; 2012; Tries, S., Nebel, S., Blanke, K.
- Comparability of Survey Measurements; 2012; Oberski, D.
- Classification of Surveys; 2012; Stoop, I., Harrison, E.
- Enhancing Web Surveys With New HTML5 Input Types; 2012; Funke, F.
- Why one should incorporate the design weights when adjusting for unit nonresponse using response homogeneity...; 2012; Kott, P. S.
- Assessing the Quality of Survey Data ; 2012; Blasius, J.
- Designing and Doing Survey Research; 2012; Andres, L.
- Using break-offs in web interviews for predicting web response in mixed mode surveys; 2011; Beukenhorst, D.
- Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 2011; 2011